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PEERING TREND 

DIRECT PEERING & MULTICLOUD 


• Traffic moving closer to users (5G, MEC, CDN) 

• 82% of Traffic will be video by 2021 

• Subscriber BW growing at 31% CAGR 

• Stricter peering SLAs with move to SaaS 


Traditional content 
peering 



O YouTube 

NETFLIX 


Optimized content 
peering 
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CISCO PEERING FABRIC DESIGN BENEFITS OVERVIEW 
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Fast Convergence 
Auto Protection Fabric 



Scalable Capacity & 
Investment Protection 
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Granular Visibility 
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Through 
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CISCO PEERING FABRIC DESIGN VALUE PROPOSITION 


• Dense, price-competitive & space/power-efficient Cisco Peering Platform. 
Cost Optimized • NSO Automated configuration & validation. 

• Maximize CAPEX network utilization with Segment Routing. 


Time to Revenue 

• Verified & tested Cisco design to shorten deployment cycle leveraging best practices. 

• Faster service activation across infrastructure with NSO & Yang models. 


Scalability & 
Reliability 

• Investment protection - fabric allowing gradual service expansions without forklifting. 

• Fabric isolates failure domains minimizing traffic impact with single node downtime. 

• Telemetry unlocks proactive fault detection & remediation. 


Security 


• Dedicated nodes isolate malicious attacks. 

• Distributed architecture gives more resources for security & monitoring 

• DDoS Protection using BGP Flowspec 


Customer 

Experience 

Cl SCO 



Fast convergence. 

Cost/power/space - optimized enabling peering localization closer to customers. 
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CISCO PEERING FABRIC DESIGN 


Overview Design Options Automation Segment Routing Best Practice 


Design Building Blocks 


End-to-End Design 


Cisco Hardware 




Lower TCO with 
Scalable Cisco 
Peering Platform 


Segment Routing 



Simplified Path 
Programmability 


Programmability, 
Telemetry & NSO 
Ansible Playbooks 


Configuration 
Best Practices 




Control-plane, 
Security & Telemetry 
Validated Design. 
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CISCO PEERING FABRIC DESIGN 


Overview Design Options Automation Segment Routing Best Practice 

Option 1: Integrated Option 2: Distributed 

OTT1 IXP1 IXP2 IXP3 OTT2 



Resiliency 

Peering 

Visibility 


Single router downtime impacts 50% of traffic 

Security & monitoring resources limited to 2 nodes 


• Router downtime impacts 25% of traffic & decreases as fabric scales. 

• Security & monitoring resources scale with leafs. 


Simplify & • Same nodes used for Internet, voice & video. 

Prioritize 


Separate peer types (ex. dedicated PE leafs for high priority traffic.) 
Simplified feature set on peering leafs. 


Investment * Future issues with line card compatibility. 

Protection * Future higher density might require forklifting. 


Each leaf independent. Future generations seamlessly added to fabric. 
Horizontal scaling saves investments. 


CAPEX & 
OPEX 


• Saves costs of optics between leafs and spines. 

• Less devices to operate. 


Less critical devices to operate & hence service costs reduced. 
Less critical software upgrades enabling faster innovation. 








CISCO PEERING FABRIC DESIGN 


Overview 


Design Options 


Automation Segment Routing 


Automation for Common Tasks 

❖ NSO Automation 

❖ OpenConfig YANG models 




Network Visualization Analytics 
Network Optimization Capacity Planning 
Anomaly Detection 


Best Practice 
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Day 0 


Day 1 


Day 2 




Initial device 
provisioning 
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• Control-plane configuration. 


• New peer turn-up and management. 

• Security ACLs. 


• Peer capacity expansion. 

• Netflow config & collector. 


• PFL fabric node turn-up. 

• Telemetry data & collector. 


• Intelligent Peering Traffic Control 

• BMP receivers. 


• Automated GW Policy Motion 


• Resiliency with Prioritization DEMO 




) 2014 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 






















CISCO PEERING FABRIC DESIGN 


Overview Design Options Automation Segment Routing Best Practice 

Service providers have been facing the following issue in peering networks: 

o Problem: Engineering the best network exit path that is cost-efficient while providing good user 
experience metrics (latency, link utilization & traffic loss). 

o Solution: Segment Routing Egress Peer Engineering (EPE). 


Segment Routing EPE enables Service Providers to dynamically adapt to the best path within their network 
and the optimal exit, based on the changing network environment. 

Optimal exit link chosen: 

• Low cost (private peer) 

Advertising: i r • Low utilization link 

10.0.0.0/24 
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CISCO PEERING FABRIC DESIGN 


Overview Design Options Automation Segment Routing Best Practice 


Peering configuration best practices that have been tested & validated to ease customer deployments. 


Configuration Building Blocks 


Base Config 

(Day 0, FIB 
optimization & 
LLDP) 


Routing 

Protocols 

(IGP, BGP, 
SR) 


Telemetry & 
Netflow 


QoS Policy 
Security ACLs 


Method Use Cases 


Netflow v9 / 

IPFIX 

• Capacity planning 

• DDoS detection 

• Peer traffic anomalies 

• Enhanced operations 

Model-Driven 
Telemetry 

• Capacity planning 

• Anomaly detection 

• Automated operational state retrieval 

BGP Monitoring 
Protocol 

• BGP prefix anomalies 

• BGP policy change impact analysis 

• BGP update statistics 

• Root-cause analysis for BGP prefix issues 
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CISCO 


Demo 

Peering Fabric Automation: 



Intelligent Peering Traffic Control 
Automated GW Policy Motion 
Resiliency with Prioritization 





Multi-Cloud and Internet Network 



SP Network 



Application 
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SP Network 




CDN GW-4 


CDN GW-2 



DEMO 



Current Challenge and Complexity: How to steer Specific Traffic to Specific Peer Link 

1. Complexity and No Flexibility of Multiple Peer Node and Link Selection 

2. Global Customer and Routing Impact for IGW Upstream Policy Change 

3. Complex Static Policy Based Routing with Performance Penalty Impact 

4. No Resiliency with Prioritization of Multiple Peer Node and Link Selection per Specific Traffic 
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LEGACY WAY TO CONTROL 
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1. Configure Complex BGP Policy & Policy Based Routing 

2. Policy Based Routing costs Performance Penalty Impact 

3. Global IGW Policy Change impact to all Customer 

4. CAN NOT STEER TE TUNNEL TO SELECTED PEER LINK 



1. Configure Complex BGP Policy & Policy Based Routing 

2. Configure Static Route & RSVP TE Tunnel 

3. Configure Complex Route Policy to use TE Tunnel 
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1b. INTELLIGENT SPECIFIC APPS TRAFFIC CONTROL 
VIA SPECIFIC PEER LINK PER SOURCE GW 
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PEER FABRIC ASN & PREFIX 
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GOOGLE & NETFLIX GW1 AS5 
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NSO 




RP/0/0/CPU0:Node-4#trace 100.100.100.100 
Sat May 11 23:03:52.232 UTC 


Type escape sequence to abort. 
Tracing the route to 100.100.100.100 


1 10.3.4.3 9 msec 0 msec 0 msec 

2 10.1.3.1 0 msec 0 msec 0 msec 

3 10.1.5.5 0 msec * 0 msec 

RP/0/0/CPU0:Node-4# 

RP/O/O/CPUO :Node-4#trace 100.100.103.103 
Sat May 11 23:04:09.101 UTC 


Type escape sequence to abort. 
Tracing the route to 100.100.103.103 
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*NETFUX^ 


RP/0/0/CPU0:Node-2#trace 100.100.100.100 
Sat May 11 23:01:30.264 UTC 


SP Network 



Type escape sequence to abort. 
Tracing the route to 100.100.100.100 


1 10.2.3.3 9 msec 0 msec 0 msec 

2 10.1.3.1 0 msec 0 msec 0 msec 

3 10.1.5.5 0 msec * 0 msec 

RP/0/0/CPU0:Node-2# 

RP/0/0/CPU0:Node-2#trace 100.100.103.103 
Sat May 11 23:02:00.732 UTC 


Type escape sequence to abort. 
Tracing the route to 100.100.103.103 


1 10.3.4.3 9 msec 0 msec 0 msec 

2 10.1.3.1 0 msec 9 msec 0 msec 

3 10.1.5.5 0 msec * 0 msec 


1 10.2.3.3 9 msec 0 msec 0 msec 

2 10.1.3.1 0 msec 0 msec 0 msec 

3 10.1.5.5 0 msec * 0 msec 
























GOOGLE : GW2 GW7 AS6 INTERNET 
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RP/0/0/CPU0:Node-2#trace 100.100.100.100 source 20.20.20.20 
Sun May 12 05:11:56.900 UTC 

Type escape sequence to abort. 

Tracing the route to 100.100.100.100 

1 10.2.3.3 [MPLS: Labels 17007/80003 Exp 0] 29 msec 19 msec 19 msec 

2 10.3.7.7 [MPLS: Label 80003 Exp 0] 9 msec 9 msec 9 msec 

3 10.6.7.6 9 msec 9 msec 9 msec 

4 10.5.6.5 9 msec * 9 msec 
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GOOGLE : GW2 GW7 AS6 INTERNET 


"ii Configuration editor /peer x + _ 1° X 

C 0 Not secure | 192.168.56.200:8080/webui-one/ConfigurationEditor/peering:peering%7BID-GW... ☆ 0 j 


• ■ 11 • 11' • Configuration editor 

CISCO VERSION: 5.1.1 


View options ▼ 


root t 


A t_/P eer ing:peering{ID-GW-2}/ 

ib 


i| »| 

CISCO 


gw* 
node-2 

advertised- network 
true 


ii © 


path Q 


□ 


destination 


google_100 


RP/0/0/CPU0:Node-2#trace www.google.com source id-cloud-1 
Sun Jun BO 09:13:51.443 UTC 

JType escape sequence to abort. 

Tracing the route to 100.100.100.100 


I 1 10.2.3.3 [MPLS: Labels 17007/80003 Exp 0] 19 msec 9 msec 9 msec 

2 10.3.7.7 [MPLS: Label 80003 Exp 0] 9 msec 9 msec 9 msec 

3 10.6.7.6 0 msec 9 msec 9 msec 

4 10.5.6.5 9 msec * 9 msec 

RP/0/0/CPU0:Node-2# 

RP/0/0/CPU0:Node-2# 

RP/0/0/CPU0:Node-2#trace www.netflix.com source id-cloud-1 
Sun Jun 30 09:14:05.312 UTC 

JType escape sequence to abort. 

(Tracing the route to 103.103.103.103 

1 10.2.3.3 9 msec 19 msec 9 msec 

2 10.1.3.1 0 msec 0 msec 0 msec 

3 10.1.5.5 0 msec * 9 msec 
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NETFLIX : GW4 GW1 AS8 INTERNET 
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RP/O/O/CPUO :Node-4#trace 100.100.103.103 source 20.20.21.21 
Sun May 12 05:13:25.542 UTC 

Type escape sequence to abort. 

Tracing the route to 100.100.103.103 

1 10.3.4.3 [MPLS: Labels 17001/80001 Exp 0] 19 msec 9 msec 9 msec 

2 10.1.3.1 [MPLS: Label 80001 Exp 0] 9 msec 9 msec 9 msec 

3 10.1.8.8 0 msec * 9 m^Pr 



























NETFLIX : GW4 GW1 AS8 INTERNET 


"i"' Configuration editor/peen x + _ 1° X 

<- -> C © Not secure | 192.168.56.200:8080/webui-one/ConfigurationEditor/peering:peering%7BID-GW... it © : 
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View options 


root 1 


A t_/P eer ing:p£ering{ID-GW-4}/ 
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path Q 
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destination 


netflix 103 


iP/0/0/CPU0:Node-4#trace www.google.com source id-cloud-, 
>un Jun 30 09:12:43.606 UTC 

rype escape sequence to abort, 
fracing the route to 100.100.100.100 


1 10.3.4.3 0 msec 0 msec 0 msec 

2 10.1.3.1 0 msec 0 msec 0 msec 

3 10.1.5.5 19 msec * 0 msec 

IP/0/0/CPU0:Node-4# 

IP/0/0/CPU0:Node-4# 

IP/0/0/CPU0:Node-4#trace www.netflix.com source id-cloud-2 
>un Jun 30 09:12:49.356 UTC 

rype escape sequence to abort, 
rracing the route to 103.103.103.103 

1 10.3.4.3 [MPLS: Labels 17001/80003 Exp 0] 9 msec 9 msec 0 msec 

2 10.1.3.1 [MPLS: Label 80003 Exp 0] 9 msec 9 msec 9 msec 

3 10.1.8.8 9 msec * 9 msec 
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RP/O/O/CPUO :Node-4#trace www.google.com source id-cloud-1 
Sun Dun 30 10:02:35.260 UTC 


INITIAL GW-4 Operational 


. 1 1 1 > 1 1 1 . Configuration editor 

CISCO VERSIONS. 1 


rt t_ /peeri n g : pe e ri n g{ ID - GW} /dom esti c /path / 


■H I | I ■ I | I ■ 

CISCO 


Type escape sequence to abort. 
Tracing the route to 100.100.100.100 


1 10.3.4.3 [MPLS: Labels 17007/80003 Exp 0] 9 msec 9 msec 9 msec 

2 10.3.7.7 [MPLS: Label 80003 Exp 0] 9 msec 9 msec 9 msec 

3 10.6.7.6 9 msec 9 msec 9 msec 

4 10.5.6.5 9 msec * 9 msec 

RP/0/0/CPU0:Node-4# 

RP/0/0/CPU0:Node-4#trace www.facebook.com source id-cloud-1 
Sun Dun 30 10:02:52.069 UTC 


Type escape sequence to abort. 
Tracing the route to 101.101.101.101 


path Q 


] destination 

path-1 

| | goocjle_100 

AS6 

H facebook_101 

ASS-7 

| | youtube_102 

AS8-1 

| | netflix_103 

AS5 


1 10.3.4.3 [MPLS: Labels 17007/80143 Exp 0] 19 msec 9 msec 9 msec 

2 10.3.7.7 [MPLS: Label 80143 Exp 0] 9 msec 9 msec 9 msec 

^ 10.7.8.8 9 msec * Q 


9 msec 





RP/0/0/CPU0:Node-4#trace www.youtube.com source id-cl 
Sun Dun 30 10:04:34.032 UTC 

Type escape sequence to abort. 

Tracing the route to 102.102.102.102 

oud-1 


1 10.3.4.3 [MPLS: Labels 17001/80003 Exp 0] 9 msec 

9 msec 

9 msec 

2 10.1.3.1 [MPLS: Label 80003 Exp 0] 9 msec 9 msec 

3 10.1.8.8 9 msec * 9 msec 

RP/0/0/CPU0:Node-4# 

9 msec 


RP/O/O/CPUO:Node-4#trace www.netflix.com source id-cloud-1 

Sun Dun 30 10:04:48.631 UTC 

Type escape sequence to abort. 

Tracing the route to 103.103.103.103 


1 10.3.4.3 [MPLS: Labels 17001/80141 Exp 0] 9 msec 

9 msec 

9 msec 

2 10.1.3.1 [MPLS: Label 80141 Exp 0] 9 msec 9 msec 

3 10.1.5.5 9 msec * 9 msec 

9 msec 
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. 1 1 1.1 1 1» Configuration editor 

CISCO VERSION: 5.1.1 
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RP/0/0/CPU0:Node-2#trace www.google.com source id-cloud 
Sun Jun 30 09:49:59.884 UTC 

Type escape sequence to abort. 

Tracing the route to 100.100.100.100 

1 10.2.3.3 [MPLS: Labels 17007/80003 Exp 0] 19 msec 19 msec 9 msec 

2 10.3.7.7 [MPLS: Label 80003 Exp 0] 9 msec 9 msec 9 msec 

3 10.6.7.6 9 msec 9 msec 9 msec 

4 10.5.6.5 9 msec * 9 msec 

RP/0/0/CPU0:Node-2#trace www.facebook.com source id-cloud 
Sun Jun 30 09:50:11.253 UTC 

Type escape sequence to abort. 

Tracing the route to 101.101.101.101 

1 10.2.3.3 [MPLS: Labels 17007/80143 Exp 0] 19 msec 9 msec 9 msec 

2 10.3.7.7 [MPLS: Label 80143 Exp 0] 9 msec 9 msec 9 msec 

3 10.7.8.8 9 msec * 9 msec 
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Automated GW Policy Motion 
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RP/0/0/CPU0:Node-2#trace www.youtube.com source id-cloud 
Sun Jun 30 09:50:22.093 UTC 

Type escape sequence to abort. 

Tracing the route to 102.102.102.102 

1 10.2.3.3 [MPLS: Labels 17001/80003 Exp 0] 19 msec 9 msec 9 msec 

2 10.1.3.1 [MPLS: Label 80003 Exp 0] 19 msec 9 msec 9 msec 

3 10.1.8.8 9 msec * 9 msec 

RP/0/0/CPU0:Node-2#trace www.netflix.com source id-cloud 
Sun Jun 30 09:50:33.392 UTC 

Type escape sequence to abort. 

Tracing the route to 103.103.103.103 

1 10.2.3.3 [MPLS: Labels 17001/80141 Exp 0] 19 msec 9 msec 9 msec 

2 10.1.3.1 [MPLS: Label 80141 Exp 0] 19 msec 9 msec 9 msec 

3 10.1.5.5 9 msec * 9 msec 
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3. Resiliency with Prioritization 







ID-CLOUD : AS65000 AS5 AS8 
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ID-CLOUD : AS65000 AS5 AS8 
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v A 


RP/O/O/CPUO :Node-6#trace id-cloud source hkg-cloud 
Sun Dun 30 10:23:38.267 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.6.7.7 9 msec 0 msec 0 msec 

2 10.3.7.3 [MPLS: Label 16299 Exp 0] 9 msec 9 msec 9 msec 

3 10.2.3.2 9 msec * 19 msec 
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ID-CLOUD : AS65000 AS5 AS8 
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RP/O/O/CPUO :Node-6#trace id-cloud source hkg-cloud 
Sun Hun 30 10:31:47.144 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.5.6.5 0 msec 0 msec 0 msec 

2 10.1.5.1 0 msec 0 msec 19 msec 

3 10.1.3.3 [MPLS: Label 16299 Exp 0] 9 msec 9 msec 9 msec 

4 10.2.3.2 9 msec * 9 msec 
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RP/0/0/CPU0:Node-6#trace id-cloud source hkg-cloud 
Sun Jun 30 10:33:47.186 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.6.8.8 9 msec 0 msec 0 msec 

2 10.1.8.1 0 msec 0 msec 0 msec 

3 10.1.3.3 [MPLS: Label 16299 Exp 0] 19 msec 19 msec 9 msec 

4 10.2.3.2 19 msec * 9 msec 
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4. Intelligent Traffic Control 
Ingress Peering Engineering 







INGRESS BASED ON NORMAL BGP 
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Status codes: s suppressed, d damped, h history, * valid, > best 

i - internal, r RIB-failure, S stale, N Nexthop-discard 
Origin codes: i - IGP, e - EGP, ? - incomplete 
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INTENT - INGRESS POLICY for ID-CLOUD ONLY 
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INTENT - INGRESS POLICY for ID-CLOUD ONLY 
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RP/0/0/CPU0:Node-5#trace id-cloud source as5-cloud 
Sun lun 30 11:07:41.101 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.5.6.6 9 msec 0 msec 0 msec 

2 10.6.7.7 0 msec 9 msec 0 msec 

3 10.3.7.3 [MPLS: Label 16299 Exp 0] 9 msec 9 msec 9 msec 

4 10.2.3.2 9 msec * 9 msec 
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INTENT - INGRESS POLICY for ID-CLOUD ONLY 
SECOND INGRESS 
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RP/0/0/CPU0:Node-5#trace id-cloud source as5-cloud 
Sun Jun 30 11:11:17.906 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.5.8.8 9 msec 0 msec 0 msec 

2 10.7.8.7 0 msec 0 msec 0 msec 

3 10.3.7.3 [MPLS: Label 16299 Exp 0] 9 msec 9 msec 9 msec 

4 10.2.3.2 9 msec * 9 msec 
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INTENT - INGRESS POLICY for ID-CLOUD ONLY 
THIRD INGRESS 
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RP/0/0/CPU0:Node-5#trace id-cloud source as5-cloud 
Sun Jun 30 11:14:37.752 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.5.8.8 9 msec 0 msec 0 msec 

2 10.1.8.1 0 msec 0 msec 0 msec 

3 10.1.3.3 [MPLS: Label 16299 Exp 0] 9 msec 9 msec 9 msec 

4 10.2.3.2 9 msec * 29 msec 
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INTENT - INGRESS POLICY for ID-CLOUD ONLY 
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RP/0/0/CPU0:Node-5#trace id-cloud source as5-cloud 
Sun Jun 30 11:15:02.080 UTC 

Type escape sequence to abort. 

Tracing the route to 20.20.22.22 

1 10.1.5.1 9 msec 0 msec 0 msec 

2 10.1.3.3 [MPLS: Label 16299 Exp 0] 9 msec 9 msec 9 msec 

3 10.2.3.2 9 msec * 9 msec 
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KEY TAKEAWAY 


Cisco Intelligent Peering Solution 

• Simple, scalable, resilient and future proof 

• Segment Routing and TI-LFA 

• Peering Fabric Design 

• Analytic and Automation 

• Cost Optimized Solution 

• Reduce transport costs 

• Traffic engineering optimizes network utilization 

• Automation reduces OpEx 

• Low power systems reduces OpEx 

• Flexible consumption model reduces Capex 

• Trust and Secure 

• Security Best Practice 

• BGP monitoring with BGP route hi-jacking detection 

• Netflow and BGP Flowspec for DDoS & Service 
. Protection 
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